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1  Summary 

In  this  work,  the  investigators  explored  the  area  of  neural-nets  for  object  recognition 
and  also  investigated  hardware  issues  in  the  optical  implementation  of  neural  nets. 
The  approach  underlying  the  design  of  networks  was  that  of  optimization  theory  to 
both  specify  the  problem  and  the  network  for  its  solution.  These  networks  differ  con¬ 
siderably  from  simple  pattern  matchers  (such  as  the  Hopfield  content-addresssable 
memory)  where  an  iconic  version  of  the  pattern  itself  is  stored.  Instead,  the  net¬ 
work  implements  a  form  of  model  matching  in  which  the  model  base  is  organized  as 
structured  graphs,  the  input  data  is  organized  ,  again  by  the  optimization  procedure, 
into  similarly  structmed  graphs  by  the  neural  net,  and  recognition  is  accomplished 
by  a  form  of  graph  matching.  The  model  base  is  organized  hierarchically  in  that 
both  compositional  (whole-part)  and  specialization  (class  membership)  notions  are 
captured  by  sparse  matrices  which  serve  as  pointer  structures  in  the  objective  func¬ 
tions.  This  notion  of  design  allows  for  a  uniform  style  of  addressing  both  low-level 
visual  problems  and  more  traditional  high-level  recognition  problems.  Our  main  ex¬ 
periments  were  conducted  in  visual  domains  involving  the  recognition  of  simple  stick 
figures.  One  version  in  which  the  coding  of  input  data  into  structured  graphs  was 
done  as  a  preprocess  achieved  considerable  success.  A  more  ambitous  experiment 
in  which  this  "perceptual  organization'’  is  done  as  part  of  the  optimization  achieved 
limited  results,  but  served  to  highlight  areas  for  further  progress.  Though  learning 
was  a  subsidiary  goal  here,  progress  was  made  in  the  areas  of  CMAC  contollers  and 
delineations  of  learning  tasks  for  the  graph  matching  circuits  mentioned  above.  Also, 
earlier  work  involving  aspects  of  more  traditional  associative  memories  was  contin¬ 
ued,  with  progress  achieved  in  the  specification  of  optimal  architectures  and  objective 
functions  for  outer-product  style  memories.  Optical  implementations  were  concerned 
mainly  with  the  problem  of  implementing  ever  more  general  interconnect  patterns. 
The  main  problem  here  is  to  use  optics  to  form  a  fixed  interconnection  network  be¬ 
tween  layers  of  2-D  nodes  (neurons).  Two  means  of  using  spatial  multiplexing  to 
effect  a  4-D  interconnect  between  two  2-D  node  planes  are  used:  multifaceted  holo¬ 
grams  and  multichannel  incoherent  imaging  systems.  Nonlinearities  are  implemented 
electrically  after  detection  of  the  light  intensity.  Computer  simulations  and  exper¬ 
iments  with  e-beam  fabricated  holograms  designed  to  code  an  associative  memoiy 
and  a  winner-take-all  network  reveal  so  far  a  host  of  technical  problems  resulting  in 
a  limited  achievable  accuracy  for  the  connection  weight. 
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2  Introduction 

The  following  is  a  summary  of  progress  accomplished  over  the  first  year  of  the  two- 
year  grant  period.  A  number  of  papers,  Tech  reports,  etc.  are  referenced  at  the  end 
of  this  1-year  progress  report. 

The  overall  theme  of  the  project  was  the  investigation  of  issues  in  the  organization 
of  visual  memory.  The  project  was  pursued  at  the  theoretical  level  as  well  as  at  the 
level  of  hardware  organization,  where  issues  in  the  optoelectronic  implementation  of 
these  memorie  were  pursued. 

This  project  grew  out  of  earlier  work  in  which  an  approach  to  object  recognition 
as  an  associative  memory  problem  was  pursued.  The  big  problem  with  this  earlier 
approach  is  that  objects  must  be  recognized  regardless  of  changes  in  position,  ro¬ 
tation,  scaling,  and  a  host  of  other  deformations.  One  may  store  iconic  patterns 
directly  in  a  memory  and  design  such  invariances  directly  into  the  connection  pat¬ 
tern,  but  the  circuits  quickly  become  complicated.  We  follow  the  approach  taken 
in  traditional  computer  vision  systems  and  store  relational  models  instead  of  iconic 
models  into  the  circuit,  and  demand  that  the  input  data  itself  be  organized  into  such 
a  relational  structure  and  optimally  matched  to  the  “nearest”  model.  Relational 
models  are  designed  to  automatically  capture  the  desired  invariances;  the  perceptual 
organization  of  input  data  into  relational  structure  proceeds  simultaneously  with 
the  matching  process.  Like  some  associative  memory  approaches,  this  one  also  uses 
objective  functions  for  specification  of  the  problem  and  design  of  the  circuit.  The 
optical  implmentation  aspect  is  largely  independent  of  the  distinction  between  the 
associative  memory  approach  and  the  optimiing  graph  matching  approach.  In  each 
case,  the  main  task  is  to  use  the  capabiliies  of  optics  and  electronics  to  for  general 
intercoimect  patterns  and  implemnbt  nonlinearities. 

Below  we  list  a  summary  of  accomplishments  completed  during  the  first  year  of 
the  grant  period.  This  is  follwed  by  a  more  detailed  discussion. 

3  Summary  of  Completed  Research 

Below  is  a  succint  summary  of  accomplishments  of  the  first  year.  Each  item  is  followed 
by  a  reference  to  an  appropriate  paper  or  technical  report  that  gives  a  more  detailed 
account. 

•  We  implemented  two  networks  for  simple  object  recognition  and  performed 
analysis  and  simulation  experiments.  Each  of  these  successfully  matched  sim¬ 
ple  stick  figures  to  a  database  of  models.  It  was  able  to  find  multiple  objects 
and  specializations  of  objects.  One  version  [4]  used  an  tmconstrained  optimiza¬ 
tion  technique  for  net  dynamics;  the  other  incorporated  “Lagrange  multiplier 
neurons”  to  implement  hard  constraints  [14|.  In  each  case,  the  difficult  ta-^k 
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was  made  tractable  by  hand  coding  the  input  data  into  relational  structures 
suitable  for  matching.  The  resulting  objective  functions  were  quadratic  and  the 
net  worked  well. 

•  A  more  difficult  version  of  the  above  task  was  attempted.  Again,  the  idea  is 
recognize  simple  stick  figures,  but  now  the  network,  as  part  of  the  optimiza¬ 
tion  process,  had  to  group  input  sticks  into  potentially  meaningful  relational 
structures.  This  resulted  in  5th-order  objective  functions.  While  success  of  the 
network  was  limited,  much  was  learned.  Results  are  reported  in  [8],  [9],  and 
(6]- 

•  A  version  involving  recognition  of  3D  objects  was  completed.  This  [15]  network 
recognizes  3D-stick  figures  from  a  2D  projection. 

•  A  paper  discussing  the  general  approach  to  object  recognition  espoused  by 
us  and  collaborators  was  invited  for  publication  by  the  new  journal  Neural 
Computation  [6]. 

•  E-beam  multifacetted  holograms  for  optical  neural  net  intercocmects  were  fab¬ 
ricated  for  an  associative  memory  problem  and  a  winner-take-all  problem.  The 
optical  results  were  compared  to  results  from  a  simulation  program  written  to 
model  sources  of  error  in  the  e-beam  scheme.  This  resulted  in  some  under¬ 
standing  of  the  engineering  problems  associated  with  the  optical  interconnect 
scheme.  Results  and  analyses  are  shown  in  this  progress  report. 

•  In  support  of  the  optical  effort,  we  completed  a  study  on  performance  of  outer- 
product  associative  memories.  One  result  [l]  showed  that  versions  with  self 
interconnects  perform  better  than  those  without;  another  [2]  showed  that  a 
version  with  positive  only  interconnects  can  be  made  to  perform  well.  Both  of 
these  results  have  ramifications  for  optical  schemes. 

•  An  additional  study  on  optimal  architectures  for  outer  product  associative 
memories  was  completed  [13].  A  universal  architecture  that  makes  optimally 
efficient  use  of  hardware  is  proposed  in  this  study. 

•  Though  learning  hasn’t  been  a  major  theme  of  the  work,  some  progress  was 
made  in  initial  studies  for  learning  distance  metrics  in  the  graph  matching 
networks  mentioned  above.  In  an  unrelated  study,  a  fast,  general  purpose, 
supervised  learning  algorithm  based  on  CMAC  models  was  devised  [ll]  [12j. 
On  a  popular  test  case,  it  greatly  outperforms  backprop. 
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4  Discussion 


The  work  splits  into  three  categories:  networks  for  model  matching,  optical  imple¬ 
mentations,  and  analyses  of  associative  memory.  The  optical  work  was  carried  out 
as  a  subcontract  at  the  University  of  Arizona  and  is  discussed  in  a  self-contained 
section. 

4.1  Networks  for  model  matching 

Neural  net  tasks  for  visual  recognition  is  often  thought  of  as  a  variant  of  some  simple 
pattern  matcher,  such  as  the  the  Hopfield  associative  memory  or  a  simple  perceptron. 
These  schemes  are  limited  in  two  ways:  “objects”  are  represented  iconically  instead 
of  in  the  more  efficient  manner  of  relational  structures,  and  there  is  no  provision  for 
efficiency  in  search  by  using  notions  of  hierarchy.  Of  course,  both  of  these  ideas  are 
common  in  traditional  computer  vision,  but  here,  we  propose  a  way  of  incorporating 
these  crucial  notions  into  a  neural-net  paradigm. 

We  introduce  an  optimization  approach  for  solving  problems  in  computer  vision 
that  involve  multiple  levels  of  abstraction.  Specifically,  our  objective  functions  can 
include  compositional  hierarchies  involving  object-part  relationships  and  specializa¬ 
tion  hierarchies  involving  object-class  relationships.  The  advantage  of  hierarchical 
organization  is  that  it  makes  the  search  process  involved  in  image  interpretation  eas¬ 
ier  to  express  and  more  efficient.  The  large  class  of  vision  problems  that  can  be 
subsumed  by  this  method  includes  traditional  model  matching,  perceptual  grouping, 
dense  field  computation  (regularization),  and  even  early  feature  detection  which  is 
often  formulated  as  a  simple  filtering  operation.  This  rzuses  the  possibility  of  solving 
within  a  single  vision  system  both  low-level  and  high-level  problems  in  a  uniform 
manner. 

Our  approach  involves  casting  a  variety  of  vision  problems  as  inexact  graph  match¬ 
ing  problems,  formulating  graph  matching  in  terms  of  constrained  optimization,  and 
using  analog  neural  networks  to  perform  the  constrained  optimization.  Figs  1  and  2 
illustrate  the  basic  idea  for  a  simple  graph  matching  problem. 

Our  extension  of  graph-matching  to  model-based  object  recognition  involves  re¬ 
garding  one  of  the  graphs  as  a  “model”  graph,  which  is  supposed  to  represent  the 
knowledge  of  shapes  within  the  system,  and  the  other  graph  as  a  “data”  graph  which 
is  obtained  from  the  ctirrent  input  data  to  the  system.  The  model-side  nodes  are 
simply  called  “models”  and  data-side  nodes  are  called  “fraunes”  (denoted  i^),  which 
are  collections  of  analog  neurons  representing  parameters  of  an  object  (and  are  de¬ 
noted  Fi„  where  s  indexes  the  parameters  of  a  single  frame).  The  instantiation  of 
a  model  in  the  image  is  expressed  by  “turning  on”  a  match  neuron  Mai  between  a 
model  a  and  its  matched  Frame  F.  We  will  refer  to  such  a  network  of  frames  and 
models  as  “Frameville”.  Basic  notions  of  Frameville  are  discussed  in  Figs  3,4 ,5.6 
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Exact  Graph  Match  Via  a  TSP-like  Network 


Represent  a  graph  by  a  sparse  binary  matrix  whose  ijth  element 
is  unity  if  node  i  is  connected  to  node  j,  and  is  zero  otherwise.  Given 
two  graphs  represented  by  and  introduce  a  match  matrix 
Mq,',  where  0  <  Mai  <  1,  to  represent  the  correspondence  between 
nodes  a  and  i  . 

A  simple  objective  function  maximizes  the  number  of  local  con¬ 
sistency  rectangles  : 

Ei{M)  =  -  E  E  GapgtjMaiMpj.  (1) 

a0  ij 

while  other  terms  reflect  the  constraint  of  one-to-one  matches  be¬ 
tween  nodes: 

E.{M)  =  E(E  Mai  -  1)'  +  E(E  Mai  -  1)^  (2) 

a  i  i  a 

and  limit  the  values  of  the  match  neurons  to  the  desired  range: 

£3(M)  =  E/"“''£/'*(i)<ix.  (3) 

at 

where  g  is  a  sigmoidal  gain  function. 

Graph  matching  is  performed  by  a  gradient  descent  procedure 
on  the  combined  objective  E  =  Ei  -t  E2  -r  or  by  a.  constrained 
optimization  technique  involving  the  introduction  of  Lagrange  mul¬ 
tiplier  neurons. 
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Exact  Graph  Matching 
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An  illustration  of  a  graph  matching  problem 


Introducing  Frames 


For  model  matching,  Ga$  becomes  INAq0  denoting  part-whole 
relationships' among  models  a  and  fS.  Likewise  g{j  becomes  inaij 
denoting  part-whole  relationships  between  data  nodes. 

The  data  nodes  themselves  become  “Frames”  which  represent  vi¬ 
sual  abstractions  in  terms  of  a  few  parameters.  The  f’th  frame,  iS', 
contains  “slots”  for  analog  parameters  Fis  and  may  participate  in 
part- whole  relationships  (inaij ^  now  variable)  with  other  frames  Fj. 
The  process  of  dynamically  employing  a  frame  to  represent  an  ab¬ 
straction  required  by  data  is  called  “allocating”  the  frame. 

The  graph-matching  objective  Ei(M)  again  maximizes  the  num¬ 
ber  of  consistency  rectangles,  but  consistency  now  involves  the  vari¬ 
able  ina  links  and  model-specific  criteria  H: 

Ei{M,  ina,  F)  =  -  E  L  IMasinaijM^iMpjH^^iFi,  Fj).  (4) 

a/J  ij 


FIfr  ^ 


Figured:  Example  of  Frameville  rectangle  rule. (a)  Example  object,  a  plane,  consists 
of  parameterized  parts  fuselage  and  wings,  (b)  Shows  the  rectaingle  relationship  between 
frames  (triangles)  representing  a  wing  and  the  fuselage  of  a  plane.  Circles  denote  dynamic 
variables,  ovals  denote  models,  and  triangles  denote  frames.  For  the  plane  and  wing  models, 
the  first  few  pairameters  of  a  frame  are  interpreted  as  position,  length,  and  orientation,  (c) 
Shows  the  sibling  competition  among  parts.  The  match  variables  along  the  dotted  lines 
(A/3,9  and  M2, 7)  are  suppressed  in  favor  of  those  along  the  solid  lines  (A/2.9  and  Msj)- 
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Objective: 

Data  and  models  must  be  consistent  . . . 

Ei{M,  ina,  f")  =  -  E  E  INAa^inatjMaiM^jH°^{Fi,  Fj).  (5) 

Uf)  ij 

Constraints: 

Unique  matches  for  the  parts  of  an  object  . . . 

E  INAaisMai  -  E  inaijM^j  =  0  (6) 

a  j 

J2ina{jMai  —  =  0.  (7) 

i  0 

Unique  specialization  through  discrimination  tree  . . . 

Mai  —  12  ISAapMpi  =  0.  (8) 

fl 

M  and  ina  make  decisions  . . . 

Mai{l-Mai)  =  0 

ina{j{l  —  inaij)  =  0.  (9) 

(or  use  standard  analog  gain  term.) 
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Introduction  of  a  Specialization  Hierarchy 

Indexing  into  a  large  database  of  models  may  be  made  efficient 
by  the  introduction  of  a  specialization  hierarchy.  We  index  the 
models  (and  the  database  of  metrics  )  by  introducing  a  static 
graph  of  pointers  ISAa0  to  act  as  both  a  discrimination  network 
and  an  inheritance  hierarchy.  Note  that  property  inheritance  is 
automatically  achieved  by  allowing  the  same  frame  to  match  to  a 
model  and  just  one  of  its  specializations: 

» 

Mai  ~  IS^apMjSi  =  0.  (10) 
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The  additional  verification  of  properties  specific  to  the  specialization 
is  simply  expressed  as  additional  model-specific  constraints  involv¬ 
ing  the  parameters. 
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The  incorporation  of  a  compositional  hierarchy  and  a  specialization  hierarchy  on 
the  model  side  is  achieved  via  graph-arcs  called  INA  links  and  ISi  links  respectively. 
The  objective  function  includes  terms  representing  a  simultaneous  match  of  a  model 
to  an  object  (on  the  data  side)  and  the  parts  of  the  model  to  parts  of  the  object  in  a 
consistent  fashion.  An  objective  function  may  be  inherited  through  the  ISA  links  from 
a  model  to  its  specializations  and  there  may  be  an  incremental  objective  function  for 
each  of  the  specializations.  N\imerical  parzuneters  are  represented  by  using  analog 
neurons  and  the  verification  of  metrical  relationships  involving  these  parameters  is 
achieved  by  corresponding  consistency  terms  in  the  objective  function. 

In  order  to  perform  perceptual  organization,  the  data-side  compositional  hierar¬ 
chies  must  be  dynamic.  To  achieve  this  we  introduce  dynamic  tna  links  on  the  data 
side.  The  ina  links  connect  more  and  less  abstract  frames,  and  their  evolution  cor¬ 
responds  to  a  search  for  a  perceptual  organization  consistent  with  the  model  graph. 
Specialization  is  implicitly  achieved  by  the  simultaneous  match  of  a  frame  to  a  model 
and  to  one  of  its  ISA  specializations. 

The  next  few  figures  show  experimental  results.  We  note  that  the  “Stickville” 
simulations  of  figs  7  and  8  do  not  involve  grouping  the  input  sticks  into  data  side 
graphs;  this  is  done  by  hand  to  make  the  task  easier  for  the  network.  The  resulting 
objective  functions  are  quadratic.  As  seen  in  figure  8,  data  in  the  form  of  a  simple 
stick  figure  of  a  jet  is  presented  to  the  network.  Models  of  mammal,  plane,  jet  are 
stored.  Both  data  and  object  relations  are  encoded  in  the  coimection  strengths  of  the 
network;  in  particular  equals  a  number  measuring  the  quality  of  match 

of  data  items  t  and  j  given  that  i  is  identified  with  model  a  and  j  with  Shown  is 
the  state  of  the  match  matrix  at  intermediate  and  final  states.  Note  that  both  plane, 
and  its  /S4  specialization  (jet)  have  been  found. 

The  next  series  of  experiments  is  depicted  in  Figs  9,10,11  and  is  described  in 
extensive  detail  in  [8].  The  parameter  evaluations  are  not  precomputed  as  before, 
but  are  computed  dynamically.  Translation  invariance  is  achieved  by  using  a  specified 
form  of  the  metric.  One  tmsurprising  result  is  that  the  network  must  be  given  a  hint 
in  the  form  of  an  initial  state  in  which  an  abtraction  frame  containing  the  parameters 
of  a  main  part  is  already  successfully  matched. 

Extensive  experimentation  and  analysis  continues  with  this  domain.  A  central 
theme  that  is  ptirsued  here  is  the  following:  How  can  the  database  be  organized  so 
that  the  search  process  is  more  successful?  These  and  other  aspects  are  duscussed  in 
[8]. 

The  networks  discxissed  here  were  hmd  designed  in  that  both  the  model  base  and 
the  definition  of  the  match  metric  were  chosen  in  an  ad  hoc  manner.  It  may  be 
possible  to  improve  the  performance  of  a  Frameville  network  by  supervised  learning. 
Learning  the  database  is  quite  difficult,  but  improving  for  greater  discrimination 
may  be  possible.  A  possible  strategy  would  be  to  present  the  network  with  examples 
of  fully  matched  models  (all  match  variables  set).  The  match  metric  is  then 
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Experimental  Results:  Stickville 


We  attempt  recognition  in  Stickville,  a  simple  domain  of  con¬ 
nected  assemblages  of  linear  “sticks” .  There  are  severe  restrictions 
here: 

•  The  data  groupings  are  precomputed,  so  ina{j  are  constant. 

•  The  ina  matrix  represents  a  tree.  The  links  are  undirected. 

•  Parameters  are  precomputed.  For  an  attached  pair  of  sticks 
{inaij  =  1),  we  use  relative  size,  angle,  and  location  of  attach 
point. 

Therefore,  the  only  dynamic  variables  are  the  match  neurons  M^i 
and  the  objective  function  is  quadratic. 


The  figure  below  shows  experimental  results. 


fig- *7 
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Stickville  match  matrix  after  28  time  steps.  The  circles  represent  the  dynamic  variables 
whose  value  is  encoded  by  the  radius  of  the  shaded  portion.  The  different  columns  represent  the 
different  data  sticks,  and  the  different  rows  represent  different  models.  The  model-base  depicted 
consists  of  plane,  jet,  mammal  and  their  parts,  and  a  root  model.  The  data  consists  of  two  jets. 
The  unconstrained  (Hopfield)  update  was  used. 
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Stickville  match  matrix  achieves  the  correct  fixed-point  after  70  time  steps.  Note  that 
^  a  stick  matches  to  a  model  and  all  of  its  generalizations  (more  than  one  meuron  on  in  a  column) 
and  that  both  the  jets  have  been  found  (more  than  one  neuron  on  in  a  row) .The  unconstrained 
j  (Hopfield)  update  was  used.  p  j  ^  ^ 


Experimental  Results:  TLville 


We  experimented  with  the  full  Frameville  machinery  in  a  charac¬ 
ter  recognition  task.  The  model  base  is  a  two-level  compositional 
hierarchy  with  characters  (“T”  and  “L”)  composed  of  unit-length 
segments  in  the  manner  of  a  standard  LED  display. 

•  Frames  contain  three  slots:  the  x,  y,  9  coordinates  of  the  segment 
which  the  frame  represents. 

•  Abstraction  in  TLville:  High-level  frames  (those  matched  to  an 
entire  character)  contain  slots  for  coordinates  of  a  designated 
main  part. 

•  Terms  in  like  (x,-  —  xj  —  enforce  translation  invari¬ 

ance  through  analog  computation.  Other  invariances  may  be 
similarly  implemented. 

•  No  ISA  specialization  mechanism. 

•  To  augment  equations  6  and  7,  we  add  penalty  terms 

E  E  MaiMa'i,  E  E  E  E  inaijinaifj.  (11) 

ioc  a'^a  j0  ji  i'^i 

Thus  the  dynamic  variables  are  M^ina^  and  F  and  the  resulting 
objective  functions  are  of  order  five. 


The  following  figures  show  experimental  results. 


(a)  (b) 


(a)Inpiit  data  consists  of  unit-length  segments  oriented  horizontally  or  ver¬ 
tically.  The  task  is  translation-invariant  recognition  of  three  segments  forming  a  “T” 
junction  (e.g.  sticks  1,2,3)  or  an  ‘"L”  (e.g.  sticks  5,6,7)  amid  extraneous  noise  sticks. 
(b)Structnre  of  network.  Models  (represented  by  ovals)  occur  at  two  levels.  /-Y4  links 
are  shown  for  a  “T”.  Each  frame  (represented  by  a  triangle)  has  three  parameters:  two 
position  coordinates  and  one  orientation  coordinate.  Also  shown  are  match  and  ina  links. 
The  bold  lines  highlight  a  possible  consistency  rectangle. 
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(b) 


Match  network.  The  value  of  each  dynamical  variable  is  displayed  as  the 
relative  area  of  the  shaded  portion  of  a  circle.  The  matrix  Mjj  indicates  matches  between 
models  and  frames  at  the  low  level;  and  Alai  indicates  matches  at  the  high  level.  The 
ina  matrix  indicates  grouping  of  ten  low-level  frames  into  three  high-level  frames.  The 
parameters  of  the  high-level  frames  are  displayed  in  the  matri.x  of  linear  analog  neurons. 
The  parameters  of  the  low-level  frames  are  held  fixed  and  are  not  displayed,  (a)  Initial  st.-ue 
of  network:  All  neurons  set  to  small  random  values  except  those  corresponding  to  matchin 
of  main  parts,  (b)  Final  state  corresponds  to  successful  match  of  the  two  junctions  an 
their  parts. 
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changed.  For  example,  it  may  be  desirable  to  make  its  value  zero  or  negative  for 
positive  training  examples  and  have  it  assume  a  large  positive  value  for  negative 
examples.  The  measure  governing  the  change  of  H  might  be  the  separability  of  the 
clusters  of  energy  values  reported  by  the  analog  computation  term  for  positive  and 
negative  examples.  The  match  metric  itself  could  be  modified  by  representing  H  in 
parameterized  form  and  descending  in  the  parameter  space. 


4.2  Work  on  associative  memory  and  learning 

The  model  matching  networks  described  above  differ  from  the  more  familiar  asso- 
ciatve  memories  well  known  to  neural  netters,  but  since  associative  memory  serves 
as  a  test  problem  for  optical  implementaions,  several  problems  in  analysis  presented 
themselves.  In  addition,  we  continued  some  previous  work  in  new  types  of  associative 
memory. 

It  turns  out  that  a  Hopfield  style  ACAM  (Associative  Content  Addressable  Mem¬ 
ory)  may  be  implemented  optically  as  projection  of  a  binary  input  state  onto  a  set  of 
stored  memories  to  obtain  a  set  of  inner  products,  followed  by  summation  of  stored 
memories  weighted  by  the  innner  products,  followed  by  thresholding.  With  this  kind 
of  optical  architecture,  it  turns  out  to  be  convenient  to  use  connections  that  are 
positive  only  instead  of  bipolar,  and  to  not  restrict  onesself  to  eliminating  self  con¬ 
nections  from  the  network.  These  two  resrictions  (bipolar  nodes,  non  self  connected 
nodes)  are  present  in  the  Hopfield  model,  but  in  two  papers  [l]  [2]  we  show  how  to 
remove  them.  In  particular,  we  show  through  statistical  arguments  that  the  model 
with  self-connected  nodes  actually  works  better  than  the  one  with  without  these,  and 
that  an  all-positive  network  is  possible  if  the  threshold  point  b  selected  judiciously. 

While  the  outer-product  memories  have  been  well  studied  and  implemented  opti¬ 
cally,  there  is  reason  to  prefer  alternative  models  that  iise  simple  template  matching 
in  conjunction  with  a  layer  of  internal  decision  units  which  compete  to  perform  a 
winner-take-all  (WTA)  function.  We  refer  to  this  a  a  unary  model.  There  are  3 
reasons  for  our  interest:  WTA  networks  are  modules  of  the  model-match  networks 
described  earlier,  they  constitute  an  implementation  challenge  for  the  optical  effort 
(see  next  section),  and  unary  models  are  interesting  in  their  own  right. 

With  this  latter  reason  as  motivation,  we  completed  a  study  of  unary  models 
[13]  and  showed  the  following:  We  present  a  universal  architecture  for  standartd 
auto-associative  memory  models  which  makes  optimally  efficient  use  of  hardware. 
This  architecture  is  described  by  a  bilinear  energy  functiom.  Bot  outer-product 
and  unary  models  can  be  viewed  as  special  cases  of  this  unversal  architecture.  The 
universal  architecture  uses  only  the  minimal  number  of  binary  connections  required 
by  information  theory  to  encode  the  stored  memories.  For  higher  order  outer  product 
memories,  the  need  for  large  numbers  of  internal  “product  units”  is  eliminated. 

In  the  previous  section,  an  approach  to  learning  in  Frameville  was  discussed  but 
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no  committment  to  a  given  learning  algorithm  was  mentioned.  Reseach  on  learning, 
though  somewhat  peripheral  to  the  immmediate  goals  in  this  contract,  was  con¬ 
ducted.  In  [11]  [12],  a  learning  algorithm  in  whuich  a  system  learns  to  approximate 
mappings  by  constructing  an  interpolating  lookup  table  on  a  lattice  of  points  in  the 
input  space. 
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4.3  Work  on  optical  implementation 

This  section  describes  work  on  optical  implinentation  performed  as  a  subcontract 
at  the  University  of  Arizona.  This  progress  report  was  written  separately  by  the 
subcontract  investigators.  Figure  numbers  refer  to  figures  within  this  report. 


AFOSR  Pr-ogr-ess  Report: 

Object: 

The  object i'..'e  of  this  portion  of  the  research  program  is  to  inuestigate 
and  develop  the  imiplementation  of  neural— netujork  architectures  via  opt¬ 
ical  technology.  The  strength  of  optics  for  this  application  lies  in  its 
•ability  to  perforrri  communication,  and  so  the  primary  effort  is  directed 
at  implementation  of  the  massive  interconnections  required  of  neural 
network. 

Approach : 

The  general  approach  to  model— b.ased  recognition  of  visual  patterns  is 
that  of  optimization  via  a  neural  network.  In  the  sirriplest  formulation  of 
this  type  of  problem.,  the  interconnection  strengths  are  known  and  fixed. 
Thus,  it  is  feasible  to  implement  the  interconnections  via  a  fixed  nona— 
daptive  structure.  In  terms  of  optical  imiementation,  this  fi>aed  struc¬ 
ture  is  easier  to  realize  than  an  adaptive  one,  and  so  the  present 
effort  is  directed  at  exploration  of  the  capabilies  and  limitations  of 
fixed  optical  interconnects.  A  fundamental  issue  concerns  the  number  of 
nodes  and  the  number  of  interconnections  that  will  be  required  for 
realistic  problems.  Ne  have  chosen  to  consider  nodes  I  aged  out  on  a  2— D 
array  to  maximiize  the  number  of  nodes  that  can  be  realized.  The  asso¬ 
ciated  4— D  interconnection  matrix  is  implemented  via  spatial  multiplex¬ 
ing,  essentially  producing  a  2—11  array  of  2— D  interconnections  all 
ecoded  on  one  large  2— D  media. 

Two  methods  of  optical  interconnection  are  being  investigated,  multifa— 
cetted  holograms  and  multichannel  incoherent  imaging  systems.  In  the 
former,  a  hologram,  hereafter  referred  to  as  a  subhologram,  is  dedi¬ 
cated  to  each  node  to  encode  the  connection  pattern  from  that  node  to 
•all  the  other  nodes  in  the  system.  A  2—11  array  of  these  subholograms 
forms  the  composite  hologram.  The  use  of  Fourier  transform  holograms 
•allows  a  simple  encoding,  where  communication  to  a  particular  node  is 
associated  with  a  particular  spatial  frequency  'Cgrating>  in  the  subholo— 
gram  and  is  independent  of  the  subhologram's  spatial  position  in  the 
composite  hologram.  The  hologram  can  be  designed  via  any  of  a  number 
of  binary  encoding  schemes  and  is  fabricated  via  e— beam  lithography. 

In  the  multichannel  imaging  system  approach,  there  are  two  geometries 
In  one,  like  the  subhologram  idea,  a  subtransparency  encodes  the  con¬ 
nection  pattern  from  a  particular  node  to  the  entire  node  array,  and  an 
•array  of  these,  one  for  each  node,  meshed  together  in  a  2— D  array 
forms  the  composite  transparency.  An  array  of  irri-aging  systems,  one  fcr 
each  node,  project  the  individual  subtransparencies  to  the  common  node 
input  plane.  This  array  of  imaging  systems  can  be  implemented  either 
via  a  lens  let  array  or  via  a  simple  shadouj— casting  optical  system.  In 
the  other  geomietry,  the  multichannel  imaging  system  <lenslet  array!’ 
forms  a  spatially  replicated  image  of  the  node  array.  Each  subtranspar¬ 
ency  encodes  the  connections  fron'i  all  the  nodes  to  a  particular  node 
Again  an  array  of  these  subtransparencies  forms  the  composite  tran¬ 
sparency.  Integration  of  the  light  errierging  from  a  subtransparency  rep¬ 
resents  the  value  of  a  node  input. 

Implementation  of  the  nonlinear  node  function  directly  via  optics  is  dit- 


*  . 


ficult.  A  dedicated  electro — optical  node  plane,,  consistinc^  of  an  annai.,)  of 
optical  detectors,,  nonlinear  amplifiers,,  and  light  sources,,  appears  to  be 
the  best  approach.  The  light  sources  can  be  either  directly  driuen  laser 
diodes  or  a  spatial  light  modulator.  Difference  amplifiers  can  be  incor¬ 
porated  in  the  electronic  chain  if  bipiolar  signals  are  required.  This 
type  of  structure  is  compatible  unth  any  of  the  Ofiitical  interconnection 
methodologies  being  considered.  In  our  ujcirk,.  the  node  function  is  pef 
formed  '..'ia  uideo  detect iori>  digital  processing  of  the  digitized  uideo 
signal.,  and  display  on  a  oideo  monitor.  For  holographic  interconnection,. 

•a  liquid  crystal  light  ualue  is  used  to  conuert  the  Mi  dec  display  into  a 
coherent  ampl  itude— modulated  pi  ane— wave. 

Hccomp  I  i  shments : 

A  simple  assoc i at i Me  memory  problem  has  been  defined  to  test  the  oper¬ 
ation  and  perforiTiance  of  the  holographic  interconnection  method.  Three 
states,  an  A,,  a  E,.  and  an  ,  are  stored  in  a  simple  autoassociatiMe 
memiory  structure.  The  node  field  consists  of  an  Sx'S  array  of  binary 
M-alued  nodes.  The  connection  hologram  consists  of  an  3;<8  array  of 
subholograms,,  each  encoding  the  64  interconnections  fromi  its  node  to 
the  other  nodes.  Hetuiorks  Luith  both  bipolar  and  unipolar  connection 
strengths  ujere  inuesti gated.  A  detailed  statistical  analysis  reueals  that 
a  systerri  uiith  unipolar  interconnections  can  actually  perforrri  better 
than  one  ujith  bipolar  interconnections  <see  preprint  "Statistical  Perfor¬ 
mance  of  Outer— Product  AssociatiMe  F'lemory  Models"  submitted  to  Applied 
Optics>. 

A  winner — take— all  c'WTA.:'  r'letwor-k  was  also  desigried.  This  network  fir'tds 
the  ma.xirrium  of  16  analog  i  iputs.  Agair'i  ari  SxrS  field  of  nodes  is  used. 
Sixteeri  riodes  ar*e  used  as  iriput  riodes,.  sixteer*!  ar*e  used  as  output 
nodes,,  and  the  r'est  are  used  to  represer'it  the  result  of  two— input  comi— 
pari  sons.  Ideally,  the  maximiurn  ar’ialog  M*alue  should  appear  ori  the  cor¬ 
responding  output  node  arid  all  other-  output  riodes  should  be  zero.  The 
inter-connections  for  this  type  of  networ-k  are  sparse  and  consists  of 
str-ictly  +1  and  —1  connections.  On  the  other-  hand,  the  analog  node  Malues 
and  the  interconnection  str-engths  ar-e  r-equir-ed  to  be  Mery  accurate  for- 
pr-oper-  functioning  of  the  network.  These  two  problems,  the  assoc i at i Me 
memory  problem  and  the  NTA  problem,  place  Mer-y  different  r-equirements 
on  the  networ-k  hardware. 

A  hologram  was  designed  and  fabricated  to  implement  the  inter-connec¬ 
tions  of  the  two  network  problems  described  aboMe.  There  ar-e  seoer-al 
steps  inMOlMed  in  designing  the  hologr-am.  For-  bipolar-  connection  pat— 
ter-ns  the  connection  rviatrix  is  split  in  two  parts,  the  positiue  par-t  and 
the  negatiMe  part.  Each  must  be  encoded  in  a  seperate  hologr-am,  the 
positiMe  part  directly  and  the  negatiue  part  with  a  sign  reuer-sal  so  thar- 
it  too  is  positiMe.  The  unipolar  connection  patter-n  ujill  be  gener-ated  by 
the  hologram  as  an  optical  intensity  distr-ibution  in  the  output  plane  ot 
the  optical  systemi.  The  hologram,  howeMer-,  pr-oduces  an  optical  ampli¬ 
tude  in  the  output  plane.  The  squar-e  r-oot  of  the  connection  pattern  sei's 
the  magnitude  that  should  be  produced.  The  phase,  howeyer,  is  ar-bitrary 
and  is  a  degree  of  fr-eedomi  inher-ent  to  the  design  process.  The  hologr- am 
tr-ansmission  should  encode  the  Four-ier-  tr-ansfor-m  of  the  comple.x  optic  al 
amplitude  in  the  output.  This  Fourier  tr-ansform  is  in  gener-al  a  comp  I  e  - 
Malued  function.  The  hologram  transmission,  howeMer-,  must  be  binary 
Malued  if  it  is  to  be  fabricated  by  e— beam  lithography.  A  number  of 


staridard  techniques  e>dst  for  this  encoding  process.  All  essentially 
convert  the  comple.x  function  into  a  real  function  by  sp-atial  carrier 
modulation.,  convert  the  real  function  into  a  positive  real  function  by 
addition  of  -a  bias,,  and  encode  the  sampled  magnitude  of  the  positive 
function  by  area  u.ieightinq  of  the  transmissive  portion  of  the  hologram. 
The  hologram  produces  the  desired  result  in  a  spatially  offset  region  of 
the  output  plane  with  a  large  on  axis  bias  term.  Our  particular  hologram 
was  encoded  as  a  binary  transmission  pattern  using  a  technique 
described  by  Dallas^. 

One  of  the  major  issues  for  optical  interconnects  is  the  achievable 
accuracy.  As  a  means  of  investigating  this  issue,,  a  number  of  individual 
test  subholograms  wihere  also  fabricated  on  the  e— beam  mask.  One  of  the 
test  holograms  encoded  the  connection  pattern  dipicted  in  fig.  1.  This 
connection  pattern  was  used  e>ctensively  in  evaluating  the  performance 
of  the  hologram. 

The  interconnection  pattern  produced  by  a  computer— generated  hologram 
is  never  perfect.  All  encoding  methods  result  in  some  degree  of  error  in 
the  resulting  connection  pattern.  Also,,  the  spatial  resolution  of  the  e— 
beam  system  sets  a  limit  to  the  dynamic  range  that  can  be  achieved  via 
the  area  encoding.  This  too  affects  the  accuracy  of  the  result.  The 
phase  function  of  the  optical  arripl itude,.  chosen  during  the  hologram 
design  process,,  influences  the  distribution  of  energy  in  the  hologram 
and  the  effective  utilization  of  the  limited  dynamic  range.  Holograms 
with  both  zero  phase  and  random  phase  i.uere  encoded.  Very  severe 
problems  can  result  when  the  finite  e>d:ent  of  the  hologram/  causes  a 
smiearing  in  the  output  plane.  Nith  random  phase  coding,,  the  smearing 
can  cause  interference  effects  that  lead  to  large  error.  The  smearing 
effect  also  causes  the  energy  frorri  the  strong  central  bias  to  spread 
into  the  area  where  the  interconnections  are  being  made.  The  optical 
distribution  in  the  output  plane  produced  by  the  zero  phase  and  the 
random  phase  holograms  are  shown  in  fig.  2a  an  2b  respectively.  Neither 
result  is  particularly  good.  Clearly^  there  is  a  problem  with  the  sm/ear— 
ing  from  the  central  bias.  The  random  phase  hologram  produces  a  higher 
dif fraction  efficiency  but  at  the  e>!pense  of  increased  error. 

Computer  simulation  of  the  hologram  performance  corroborates  the 
story  told  above.  The  comiputer  simulation  was  done  by  an  exact  Fourier 
transform  of  the  computer— generated  hologram.  A  sampled  version  of 
the  output  is  shown  in  fig.  3,.  where  fig  3a  corresponds  to  the  zero 
phase  hologram/  and  fig.  3b  to  the  random  phase  hologram.  These  can  be 
compared  directly  to  the  results  in  fig.  2.;  good  agreement  is  noted.  This 
is  encouraging  for  it  allows  the  comiputer  simulation  approach  to  be 
used  to  test  alternative  hologram  designs  without  resort  to  e.'xpensive 
and  time-consuming  fabrication.  For  example,,  one  solution  to  the  prob¬ 
lem  of  smeared  light  from  the  on— axis  bias  light  is  to  make  a  binary 
phase  hologram  from  the  binary  transmission  hologram.  A  binary  phase 
hologram  has  two  transm/ission  values  of  1  and  it  rather  than  1  and  O  It 
there  are  equal  areas  of  the  tw/o  tr.ansmissions,  the  on— axis  bias  signal 
is  identically  zero.  A  simiulation  of  this  hologram  produces  the  result 
shown  in  fig.  4;  a  clear  improvement  over  the  binary  tranmission  case 

1.  B.  R.  Frieden..  ed..  The  C  o  mp  u  t  e  r  in  Opti¬ 
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qraphic  copies  made  bq  contact  printinq.  This  approach  has  not  met  ujith 
much  success.  It  is  uerq  difficult.,  un'th  bleached  hologr-ams.,  to  obt-ain 
uniform  phase  independent  of  spatial  frequencq. 

The  full  assoc i at i'v'e  memory  network  was  .attempted  using  the  electro- 
optic  node  plane  and  the  holoqraphic  interconnect.  The  errors  in  the 
connection  p.attern  resulted  in  poor  performance  for  the  assoc i at ioe 
memorq  problem.  The  encoded  states  of  the  network  ojere  not  fullq 
stable.,  though  the  patterns  were  stable  enough  to  produce  recognizable 
characters.  We  are  still  investigating  the  behavior  of  these  opt i cal  I g 
i  nter connected  networks . 

Future  Directions: 

Several  areas  will  be  investigated  in  an  attempt  to  improve  the  per  for — 
miance  of  the  holographic  interconnects.  Apod izat ion  of  the  subholograms 
will  be  explored  as  a  means  of  confining  the  energy  and  reducing  the 
deleterious  effects  of  smearing.  AlsOj.  increasing  the  seperation  of  the 
nodes  and  the  seperation  of  the  inter .r;onnect  region  from  the  central 
bias  can  be  used  to  improve  per f orm-ance .  There  is  an  obvious  tradeoff 
here  between  the  accuracy  of  the  interconnection  and  the  number  of  in¬ 
terconnects  that  can  be  realized.  The  issue  of  accuracy  in  network  in¬ 
terconnections  is  being  studied  theoretically  in  an  attempt  to  delineate 
exactly  what  the  accuracy  reguiremients  are  for  various  kinds  of  neural 
networks . 

Further  investigation  of  the  ojhat  p<hase  function  should  be  specified  for 
the  ouput  amplitude  will  be  done.  It  is  certainly  possible  that  some  det¬ 
erministic  phase  function  is  preferable  to  either  the  zero  pkiase  of  the 
randomi  phase  solutions.  We  tuill  also  explore  the  performance  of  various 
hologram  encoding  procedures  including  the  Lohmann  technique.,  the  Lee 
technique.,  and  the  Burch  technique.  A  related  idea  is  to  pose  the  encod¬ 
ing  as  an  optimization  problem  and  employ  a  technique  such  as  simulated 
annealing  in  an  attempt  to  find  the  "best"  solution.  This  approach  might 
be  considered  as  a  network  solution  to  the  problem  of  defining  a  holo¬ 
graphic  interconnect  for  another  netojork  problem. 

All  of  the  above  investigations  will  be  done  through  computer  simulation 
When  an  optimal  approach  is  defined.,  a  reasonably  large  network  con¬ 
sisting  of  perhaps  64x‘64  nodes  will  be  attempted. 

Another  goal  will  be  to  investigate  rriethods  of  producing  rriore  accurate 
binary  phase  holograms.  A  number  of  techniques  for  this  exist.  We  will 
investigate  a  photolithographic  technique.,  a  photolithographic  technique 
ojith  vacuun'i  deposition  of  a  dialectriii;  coating.,  and.,  if  possible.,  photoli¬ 
thography  with  reactive  ion  etching. 

Investigation  of  the  incoherent  optical  interconnection  methods  will  also 
be  done  in  the  con-iing  year.  A  high— resolution  film  u.iriter  using  laser 
scanning  technology  will  be  available  shortly.  Production  of  transpar¬ 
ency  masks  via  this  instruirtent  should  be  relatively  straightforward 


Figure  Captions: 

Fig.  1.  Pattern  of  interconnection  of  the  test  suhhologr-am.  The  connec¬ 
tion  to  the  node  in  the  loujer  left  has  a  Malue  of  5;  all  other  connec¬ 
tions  are  either  0  or  1. 

Fig.  2.  Holographic  interconnection  piattern  of  the  test  holograms  —  zero 
phase  t!.a>  and  random  phase  <fc>>.  Hote  the  severe  spreading  of  the  zero 
bias  light  into  the  interconnection  region. 

Fig.  3.  Calculated  interconnection  pattern  of  the  test  holograms  —  zero 
phase  (.'a''  and  random  phase  <;b>.  Good  agreement  ujith  the  optically  gen¬ 
erated  interconnection  patterns  <.fig.  2>  is  noted.,  apart  from  the  overall 
photogr  aph  i  c  i  ntens  i  t y . 

Fig.  4.  Calculated  interconnection  pattern  of  a  binary  phase  hologram. 
Comparison  to  the  results  of  the  binary  amplitude  hologram  tfig.  3a..' 
re'...'eals  greatly  reduced  error  from  the  bias  light  art':!  impro'..'ed  diffrac¬ 
tion  effi'oiency. 
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5  Summary  of  Personnel  and  Publications 

Below  is  a  list  of  professional  personnel  associated  with  this  grant: 

•  G.R.  Gindi,  Associate  Prof.  Diagnostic  Radiology  and  Electrical  Engineer¬ 
ing, Yale  U. 

•  A.F.  Gmitro,  Assistant  Prof.  Diagnostic  Radiology  and  Optical  Sciences,  U.  of 
Arizona 

•  John  Moody,  Associate  Research  Scientist,  Depertment  of  Computer  Science, 
Yale  University 

•  Joachim  Utans,  Graduate  Student,  Department  Electrical  Engineering,  Yale  U. 

•  Kannan  Parthasarathy,  Graduate  Student,  Department  of  Electrical  Engineer¬ 
ing,  Yale  U. 

•  Paul  Keller,  Graduate  student ,  Optical  Sciences  Center,  University  of  Arizona. 

•  Tony  Zador,  Graduate  Student,  Section  of  Neuroanatomy,  Yale  U. 

•  Grant  Shumaker,  Medical  Student,  Yale  University  School  of  Medicine 

Below  is  a  list  of  publications  resulting  from  this  grant: 

•  G.R.  Gindi,  A.F.  Gmitro,  and  K.  Parasarathy,  “Hopfield  Model  Associative 
Memory  with  Nonzero  Diagonal  Terms  in  Memory  Matrix”,  Applied  Optics, 
37,  129-135  (1988). 

•  A.F.  Gmitro,  P,  E.  Keller,  and  G.R.  Gindi,  “Statbtical  Performance  of  Outer- 
Product  Associative  Memory  Models”,  Applied  Optics,  (accepted  for  publica¬ 
tion) 

•  E.  Mjolsness,  G.  R.  Gindi,  P.  Anandan,  “Neural  Networks  for  Model  Matching 
and  Perceptual  Organization”,  1988  Neural  Information  Processing  Systems, 
(to  appear  in  book  of  proceedings) 

•  P.  Anandan,  E.  Mjolsness,  G.R.  Gindi,  “Low-level  Visual  Grouping  via  Op¬ 
timization  in  Neural  Networks”,  submitted  to  1989  Conference  on  Computer 
Vision  and  Pattern  Recognition. 

•  T.  Zador,  G.  Gindi,  E.  Mjolsness,  P.  Anandan,  “Neural  Network  for  Model 
based  Recognition  -  Simulation  Results",  Proc.  Inti.  Conference  on  Neural 
Networks,  p.532,  Boston  1988. 
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•  E.  Mjolsness,  G.  R.  Gindi,  P.  Anandan,  “Objective  Functions  for  Visual  Recog¬ 
nition:  A  Neural  Network  that  Incorporates  Inheritance  and  Abstraction”, 
Proc.  Snowbird  Conference  on  Neural  Networks  for  Computing,  Salt  Lake 
City,  Utah  1988. 

•  E.  Mjolsness,  G.R.  Gindi,  and  P  Anandan,  “Optimization  in  Model  Matching 
and  Perceptual  Organization”  to  appear  in  Neural  Computation  Vol.  1  no.  2 
May  1989 

•  E.  Mjolsness,  G.R.  Gindi,  and  P  Anandan,  Objective  Functions  in  Recogni¬ 
tion”,  AAI  Spring  Symposium  on  on  Physical  and  Biological  Approaches  to 
Computational  Vision”  (Stanford  March  1988)  pp.  105-107 

Below  is  a  list  of  abtracts  and  technical  reports  associated  with  this  gramt: 

•  J.  Utans,  G.  Gindi,  E.  Mjobness,  and  P.  Anandan’  “Neural  Networks  for  Ob¬ 
ject  Recognition  within  Compositional  Hierarchies”,  Technical  Report,  Yale 
University  Center  for  Systems  Science,  TR  8903,  February  1989. 

•  E.  Mjolsness,  G.R.  Gindi,  and  P  Anandan,  “Optimization  in  Model  Matching 
and  Perceptual  Organization”  Technical  Report  YALEU/DCS/RR-634  Yale  U. 
Dept  Computer  Science,  June  1988 

•  A.F.  Gmitro  and  P.  Keller,  “Space- Variant  Optical  Interconnects  via  Multi¬ 
faceted  Planar  Holograms”,  presented  at  Optical  Society  of  America  Annual 
Meeting,  Santa  Clara  CA,  October  1988. 

•  J.  Moody  and  C.  Darken,  “Learning  with  Localized  Receptive  Fields”,  Tech 
Report  YALEU/DCS/RR-649,  Yale  U.Dept  Computer  Science,  Sptember  1988. 

•  J.  Moody  and  C.  Darken,  “Fast  Learning  in  Networks  of  Locally-Tuned  Pro¬ 
cessing  Units”,  Tech  Report  YALEU/DCS/RR-654,  Yale  U.Dept  Computer 
Science,  Sptember  1988. 

•  J.  Moody,  “Optimal  Architectures  and  Objective  Ftmctions  for  Associative 
Memory:  A  Summary”  internal  report  (available  on  request)  Yale  U.Dept  Com¬ 
puter  Science,  January  1989. 

•  G.  Shumaker,  G.  Gindi, “Stickville”,  Technical  Report,  Yale  University  Center 
for  Systems  Science,  (in  preparation) 

•  G.  Shumaker,  “A  Stickville  calable  of  Rotational  Projection:  Rotatorberg  " . 
Technical  Report,  Yale  University  Center  for  Systems  Science,  (in  preparation! 
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